1. Technical Field
The present invention relates to computer servers in general, and, in particular, to a method and apparatus for managing read/write requests within computer servers. Still more particularly, the present invention relates to a method and apparatus for coalescing acknowledge packets within a computer server.
2. Description of Related Art
Infiniband provides a hardware message passing mechanism that can be used for inter-process communications between computer nodes. Typically, a consumer can access the Infiniband message passing mechanism by posting send/receive messages to send/receive work queues on a host channel adapter. The consumer can retrieve the results of these messages from a completion queue.
A host channel adapter at a source node segments outbound messages and then sends the outbound messages to a destination node. A host channel adapter at a destination node reassembles inbound messages and then places the inbound messages in a memory space designated by a consumer of the destination node.
Reliable connected service requires that Acknowledge (Ack) or Response packets be sent by a Responder device for received request packets. A requester correlates Ack/Response packets to it's outstanding requests using packet sequence numbers (PSNs). The PSN of Ack/Response packets must be kept in the same order as the original requests. However, the Infiniband architecture allows coalescing of multiple Ack/Response packets into a single Ack/Response packet, which reduces the number of packets on the link such that bandwidth will be improved. For example, if a Responder device receives Write Requests with PSNs of 2, 3 and 4, the Responder device could send a single Ack packet with a PSN of 4.
For obvious reasons, responses to Read Requests cannot be coalesced; however, a Read Response can coalesce Write Request Acks. For example, if a Responder device receives Write Request Packet PSN=1, Read Request Packet PSN=2, and Write Request Packet PSN=3 in that order, the Responder device may respond with Read Response PSN=2 and Ack PSN=3. Write Packet PSN=1 is implicitly acknowledged by the Read Response of PSN=2.
In order to meet the goal of improving link bandwidth by reducing the number of Ack packets, special considerations in the design of a Responder device are required. In general, it is desirable to coalesce Ack packets for Write Request packets when the outgoing link is busy, and send the Acks packets when the link is idle. However, the rules of coalescing must still be observed; that is, each Read Response must be sent explicitly and cannot be coalesced.
In addition, it is essential that deadlocks be avoided. Deadlocks arise when the reception of an incoming packet is prevented due to the inability to transmit a packet. Therefore, a mechanism is needed to allow an “essentially infinite” number of Write Request packets to be received and have Ack packets scheduled for them without requiring the Ack packets to be transmitted. The greater the independence between the receiving side and the transmitting side of a Responder device, the better the performance and the easier to avoid deadlock conditions.
One known method of accomplishing the above-mentioned objectives in a Responder device is to use a separate Read Request queue to hold received Read Requests waiting for execution, and an independent Ack/Response queue for response packets waiting for transmission. But with such arrangement, the allowance of Read Responses to coalesce Ack packets becomes a fairly complex problem. The above-mentioned arrangement also makes it difficult to ensure that Ack packets and Read Responses will be transmitted in a correct PSN order.
Consequently, it would be desirable to provide an improved method and apparatus for coalescing Ack/Response packets within a computer server.